Overview of this tutorial

After going through this tutorial you should be able to:

  1. Read 10X single-cell RNA-seq data into SingleCellExperiment objects
  2. Know how to navigate SingleCellExperiment objects
  3. Basic QC of single-cell RNA-seq data using scater
  4. Create low dimensional plots (PCA, UMAP, tSNE)
  5. Assign cells to known cell types using cellassign

Required packages

suppressPackageStartupMessages({
  library(scater) # BioConductor
  library(SingleCellExperiment) # BioConductor
  library(DropletUtils) # BioConductor
  library(tidyverse) # CRAN
  library(here) # CRAN
  library(DT) # CRAN
  library(pheatmap) # CRAN
})
Warning: package 'tibble' was built under R version 3.5.2
Warning: package 'purrr' was built under R version 3.5.2
Warning: package 'pheatmap' was built under R version 3.5.2
knitr::opts_chunk$set(echo = TRUE,
                      cache = TRUE)

If you need to install any of these, you can do so by:

install.packages(c("tidyverse", "here", "DT", "pheatmap", "BiocManager"))

BiocManager::install(c("scater", "SingleCellExperiment", "DropletUtils"))

It is highly recommended you do so before the tutorial as this can take significant time!

Installation of cellassign

To install cellassign, we need to install Google’s Tensorflow framework. If this doesn’t work in the tutorial - don’t worry, you won’t need it for 80% of what we cover.

install.packages("tensorflow", repos = "http://cran.us.r-project.org")

The downloaded binary packages are in
    /var/folders/wh/6v_w5j853g11zfhlb7f2rcvw0000gn/T//RtmpievTcL/downloaded_packages
tensorflow::install_tensorflow()
Creating virtualenv for TensorFlow at  /Users/kierancampbell/.virtualenvs/r-tensorflow 
Installing TensorFlow ...

Installation complete.
install.packages("devtools", repos = "http://cran.us.r-project.org") # If not already installed

  There is a binary version available but the source version is
  later:
         binary source needs_compilation
devtools 1.13.6  2.0.1             FALSE
installing the source package 'devtools'
devtools::install_github("Irrationone/cellassign")
Skipping install of 'cellassign' from a github remote, the SHA1 (3ce59232) has not changed since last install.
  Use `force = TRUE` to force installation

Read in 10X scRNA-seq data

We’re going to use some re-processed data from Single cell RNA sequencing of human liver reveals distinct intrahepatic macrophage populations (patient 1 specifically). If you git cloned this repository (http://github.com/kieranrcampbell/https://github.com/kieranrcampbell/r-workshop-march-2019) this can be found in the directory data/outs/filtered_gene_bc_matrices/GRCh38:

data_dir <- here("data", "outs", "filtered_gene_bc_matrices", "GRCh38")

print(data_dir)
[1] "/Users/kierancampbell/ubc/r-workshop-march-2019/data/outs/filtered_gene_bc_matrices/GRCh38"
print(dir(data_dir))
[1] "barcodes.tsv" "genes.tsv"    "matrix.mtx"  

10X data typically has a barcodes file (indicating cell barcodes), a genes file (indicating a mapping between genes and indices) and the actual expression data as raw counts in matrix.mtx.

We can read these in to a SingleCellExperiment object using the read10xCounts function

sce <- read10xCounts(data_dir)

sce
class: SingleCellExperiment 
dim: 33694 1767 
metadata(0):
assays(1): counts
rownames(33694): ENSG00000243485 ENSG00000237613 ...
  ENSG00000277475 ENSG00000268674
rowData names(2): ID Symbol
colnames: NULL
colData names(2): Sample Barcode
reducedDimNames(0):
spikeNames(0):

A brief tour of the SingleCellExperiment package

Overall idea: think of your count matrix where the columns are cells and the rows are genes.

So when you see things like rowData think geneData, and colData think cellData!

Data dimensions

Getting data dimensions

nrow(sce) # Number of rows = genes
[1] 33694
ncol(sce) # Number of columns = cells
[1] 1767
print(sce)
class: SingleCellExperiment 
dim: 33694 1767 
metadata(0):
assays(1): counts
rownames(33694): ENSG00000243485 ENSG00000237613 ...
  ENSG00000277475 ENSG00000268674
rowData names(2): ID Symbol
colnames: NULL
colData names(2): Sample Barcode
reducedDimNames(0):
spikeNames(0):

Subsetting

sce[, c(1,3,5)] # Subset to cells 1, 3, 5
class: SingleCellExperiment 
dim: 33694 3 
metadata(0):
assays(1): counts
rownames(33694): ENSG00000243485 ENSG00000237613 ...
  ENSG00000277475 ENSG00000268674
rowData names(2): ID Symbol
colnames: NULL
colData names(2): Sample Barcode
reducedDimNames(0):
spikeNames(0):
sce[c(2,4,6), ] # Subset to genes 2, 4, 6 
class: SingleCellExperiment 
dim: 3 1767 
metadata(0):
assays(1): counts
rownames(3): ENSG00000237613 ENSG00000238009 ENSG00000239906
rowData names(2): ID Symbol
colnames: NULL
colData names(2): Sample Barcode
reducedDimNames(0):
spikeNames(0):

Feature and cell metadata

rownames = gene names

head(rownames(sce))
[1] "ENSG00000243485" "ENSG00000237613" "ENSG00000186092" "ENSG00000238009"
[5] "ENSG00000239945" "ENSG00000239906"
ENSG00000243485

ENSG00000237613

ENSG00000186092

ENSG00000238009

ENSG00000239945

ENSG00000239906

colnames = cell names = barcodes (sometimes)

head(colnames(sce))
NULL

Column data (cell specific metadata)

head(colData(sce))
DataFrame with 6 rows and 2 columns
                                                                                      Sample
                                                                                 <character>
1 /Users/kierancampbell/ubc/r-workshop-march-2019/data/outs/filtered_gene_bc_matrices/GRCh38
2 /Users/kierancampbell/ubc/r-workshop-march-2019/data/outs/filtered_gene_bc_matrices/GRCh38
3 /Users/kierancampbell/ubc/r-workshop-march-2019/data/outs/filtered_gene_bc_matrices/GRCh38
4 /Users/kierancampbell/ubc/r-workshop-march-2019/data/outs/filtered_gene_bc_matrices/GRCh38
5 /Users/kierancampbell/ubc/r-workshop-march-2019/data/outs/filtered_gene_bc_matrices/GRCh38
6 /Users/kierancampbell/ubc/r-workshop-march-2019/data/outs/filtered_gene_bc_matrices/GRCh38
             Barcode
         <character>
1 AAACCTGCAGTAAGCG-1
2 AAACCTGGTCCGTTAA-1
3 AAACCTGGTTCTGTTT-1
4 AAACCTGTCCTCATTA-1
5 AAACGGGAGTAGGCCA-1
6 AAACGGGAGTGTACTC-1
<character>

/Users/kierancampbell/ubc/r-workshop-march-2019/data/outs/filtered_gene_bc_matrices/GRCh38

/Users/kierancampbell/ubc/r-workshop-march-2019/data/outs/filtered_gene_bc_matrices/GRCh38

/Users/kierancampbell/ubc/r-workshop-march-2019/data/outs/filtered_gene_bc_matrices/GRCh38

/Users/kierancampbell/ubc/r-workshop-march-2019/data/outs/filtered_gene_bc_matrices/GRCh38

/Users/kierancampbell/ubc/r-workshop-march-2019/data/outs/filtered_gene_bc_matrices/GRCh38

/Users/kierancampbell/ubc/r-workshop-march-2019/data/outs/filtered_gene_bc_matrices/GRCh38

<character>

AAACCTGCAGTAAGCG-1

AAACCTGGTCCGTTAA-1

AAACCTGGTTCTGTTT-1

AAACCTGTCCTCATTA-1

AAACGGGAGTAGGCCA-1

AAACGGGAGTGTACTC-1

I might want to set the column names to the barcode:

colnames(sce) <- colData(sce)$Barcode

head(colnames(sce))
[1] "AAACCTGCAGTAAGCG-1" "AAACCTGGTCCGTTAA-1" "AAACCTGGTTCTGTTT-1"
[4] "AAACCTGTCCTCATTA-1" "AAACGGGAGTAGGCCA-1" "AAACGGGAGTGTACTC-1"
AAACCTGCAGTAAGCG-1

AAACCTGGTCCGTTAA-1

AAACCTGGTTCTGTTT-1

AAACCTGTCCTCATTA-1

AAACGGGAGTAGGCCA-1

AAACGGGAGTGTACTC-1

And I can subset on barcode:

sce[, "AAACCTGCAGTAAGCG-1"]
class: SingleCellExperiment 
dim: 33694 1 
metadata(0):
assays(1): counts
rownames(33694): ENSG00000243485 ENSG00000237613 ...
  ENSG00000277475 ENSG00000268674
rowData names(2): ID Symbol
colnames(1): AAACCTGCAGTAAGCG-1
colData names(2): Sample Barcode
reducedDimNames(0):
spikeNames(0):

Row data (gene specific metadata)

head(rowData(sce))
DataFrame with 6 rows and 2 columns
                             ID        Symbol
                    <character>   <character>
ENSG00000243485 ENSG00000243485  RP11-34P13.3
ENSG00000237613 ENSG00000237613       FAM138A
ENSG00000186092 ENSG00000186092         OR4F5
ENSG00000238009 ENSG00000238009  RP11-34P13.7
ENSG00000239945 ENSG00000239945  RP11-34P13.8
ENSG00000239906 ENSG00000239906 RP11-34P13.14
<character>

ENSG00000243485

ENSG00000237613

ENSG00000186092

ENSG00000238009

ENSG00000239945

ENSG00000239906

<character>

RP11-34P13.3

FAM138A

OR4F5

RP11-34P13.7

RP11-34P13.8

RP11-34P13.14

reducedDims is where our PCA,UMAP,tSNE representations will live - but we haven’t made them yet

reducedDims(sce)
List of length 0
names(0): 

sizeFactors is where our cell size factors will live - but we haven’t calculated them yet

head(sizeFactors(sce))
NULL

The ability to have multiple assays is one of the unique advantages of the SingleCellExperiment approach - I can carry around counts, logcounts, and any other weird data transformation I might like. Right now we only have raw counts, because that’s what we’ve read in:

names(assays(sce))
[1] "counts"
counts
assay(sce, "counts")
counts(sce)

I can make my own:

assay(sce, "my_super_strange_assay") <- sin(as.matrix(counts(sce)))

names(assays(sce))
[1] "counts"                 "my_super_strange_assay"
counts

my_super_strange_assay
class(assay(sce, "my_super_strange_assay"))
[1] "matrix"
matrix

Note the beauty of SingleCellExperiments is that subsetting is consistent: if I want to subset only some cells and genes:

sce_subset <- sce[c(1,3,5), c(2,4,6,8)]

Then everything else is subset too!

print(dim(counts(sce_subset)))
[1] 3 4
print(length(sizeFactors(sce_subset)))
[1] 0
print(dim(rowData(sce_subset)))
[1] 3 2

So the approach may seem like a lot of work up front to just carry around a count matrix and some metadata, but this sort of consistent subsetting makes it much harder (but still not impossible) to introduce bugs into your analysis.

Quality control of scRNA-seq data

Getting started

First we do some key things to our data:

  1. Get some extra gene data, including the chromosome name
  2. Compute the size factors
  3. Compute normalized log counts
rowData(sce)$ensembl_gene_id <- rownames(sce)
  
sce <- getBMFeatureAnnos(sce, 
                         filters = "ensembl_gene_id",
                         attributes = c("ensembl_gene_id", "hgnc_symbol", "entrezgene",
                                        "start_position", "end_position", "chromosome_name"),
                         dataset = "hsapiens_gene_ensembl")
Batch submitting query [>-------------------------] 3% eta: 1m Batch
submitting query [>-------------------------] 4% eta: 45s Batch
submitting query [=>------------------------] 6% eta: 40s Batch
submitting query [=>------------------------] 7% eta: 47s Batch
submitting query [=>------------------------] 9% eta: 43s Batch
submitting query [==>-----------------------] 10% eta: 43s Batch
submitting query [==>-----------------------] 12% eta: 40s Batch
submitting query [==>-----------------------] 13% eta: 39s Batch
submitting query [===>----------------------] 15% eta: 38s Batch
submitting query [===>----------------------] 16% eta: 36s Batch
submitting query [====>---------------------] 18% eta: 34s Batch
submitting query [====>---------------------] 19% eta: 33s Batch
submitting query [====>---------------------] 21% eta: 32s Batch
submitting query [=====>--------------------] 22% eta: 31s Batch
submitting query [=====>--------------------] 24% eta: 30s Batch
submitting query [=====>--------------------] 25% eta: 29s Batch
submitting query [======>-------------------] 26% eta: 28s Batch
submitting query [======>-------------------] 28% eta: 27s Batch
submitting query [=======>------------------] 29% eta: 27s Batch
submitting query [=======>------------------] 31% eta: 26s Batch
submitting query [=======>------------------] 32% eta: 25s Batch
submitting query [========>-----------------] 34% eta: 24s Batch
submitting query [========>-----------------] 35% eta: 24s Batch
submitting query [=========>----------------] 37% eta: 25s Batch
submitting query [=========>----------------] 38% eta: 24s Batch
submitting query [=========>----------------] 40% eta: 23s Batch
submitting query [==========>---------------] 41% eta: 23s Batch
submitting query [==========>---------------] 43% eta: 22s Batch
submitting query [==========>---------------] 44% eta: 23s Batch
submitting query [===========>--------------] 46% eta: 24s Batch
submitting query [===========>--------------] 47% eta: 24s Batch
submitting query [============>-------------] 49% eta: 24s Batch
submitting query [============>-------------] 50% eta: 25s Batch
submitting query [============>-------------] 51% eta: 25s Batch
submitting query [=============>------------] 53% eta: 24s Batch
submitting query [=============>------------] 54% eta: 23s Batch
submitting query [==============>-----------] 56% eta: 22s Batch
submitting query [==============>-----------] 57% eta: 21s Batch
submitting query [==============>-----------] 59% eta: 20s Batch
submitting query [===============>----------] 60% eta: 19s Batch
submitting query [===============>----------] 62% eta: 18s Batch
submitting query [===============>----------] 63% eta: 18s Batch
submitting query [================>---------] 65% eta: 17s Batch
submitting query [================>---------] 66% eta: 16s Batch
submitting query [=================>--------] 68% eta: 16s Batch
submitting query [=================>--------] 69% eta: 15s Batch
submitting query [=================>--------] 71% eta: 14s Batch
submitting query [==================>-------] 72% eta: 13s Batch
submitting query [==================>-------] 74% eta: 12s Batch
submitting query [===================>------] 75% eta: 12s Batch
submitting query [===================>------] 76% eta: 11s Batch
submitting query [===================>------] 78% eta: 10s Batch
submitting query [====================>-----] 79% eta: 10s Batch
submitting query [====================>-----] 81% eta: 9s Batch
submitting query [====================>-----] 82% eta: 8s Batch
submitting query [=====================>----] 84% eta: 8s Batch
submitting query [=====================>----] 85% eta: 7s Batch
submitting query [======================>---] 87% eta: 6s Batch
submitting query [======================>---] 88% eta: 5s Batch
submitting query [======================>---] 90% eta: 5s Batch
submitting query [=======================>--] 91% eta: 4s Batch
submitting query [=======================>--] 93% eta: 3s Batch
submitting query [=======================>--] 94% eta: 3s Batch
submitting query [========================>-] 96% eta: 2s Batch
submitting query [========================>-] 97% eta: 1s Batch submitting
query [=========================>] 99% eta: 1s Batch submitting query
[==========================] 100% eta: 0s
# Calculate size factors
sce <- scran::computeSumFactors(sce, BPPARAM = MulticoreParam(10))
Warning in FUN(...): encountered negative size factor estimates
# Compute log normal expression values
sce <- normalize(sce)
names(assays(sce))
[1] "counts"                 "my_super_strange_assay"
[3] "logcounts"             
counts

my_super_strange_assay

logcounts
head(sizeFactors(sce))
[1] 0.033215558 0.071659554 0.009291482 3.408452925 2.399221881 0.106118551
sce <- runPCA(sce)
sce <- runUMAP(sce)
reducedDims(sce)
List of length 2
names(2): PCA UMAP

Just give me my PCA!

head(reducedDims(sce)[['PCA']])
                         PC1       PC2
AAACCTGCAGTAAGCG-1 -6.822310 -1.921039
AAACCTGGTCCGTTAA-1 -3.898654 -2.770704
AAACCTGGTTCTGTTT-1 -7.841220 -1.896710
AAACCTGTCCTCATTA-1  7.392807  2.167728
AAACGGGAGTAGGCCA-1 12.452015 -4.042698
AAACGGGAGTGTACTC-1 -4.069171 -1.635552

I like to add the symbols to the rownames:

rownames(sce) <- paste0(rowData(sce)$Symbol, "_", rownames(sce))
head(rownames(sce))
[1] "RP11-34P13.3_ENSG00000243485"  "FAM138A_ENSG00000237613"      
[3] "OR4F5_ENSG00000186092"         "RP11-34P13.7_ENSG00000238009" 
[5] "RP11-34P13.8_ENSG00000239945"  "RP11-34P13.14_ENSG00000239906"
RP11-34P13.3_ENSG00000243485

FAM138A_ENSG00000237613

OR4F5_ENSG00000186092

RP11-34P13.7_ENSG00000238009

RP11-34P13.8_ENSG00000239945

RP11-34P13.14_ENSG00000239906

We next need to work out which genes are mitochondrial and ribosomal as these work well for QC:

# Get Mitochondrial genes for QC:
mt_genes <- which(rowData(sce)$chromosome_name == "MT")
ribo_genes <- grepl("^RP[LS]", rowData(sce)$Symbol)
feature_ctrls <- list(mito = rownames(sce)[mt_genes],
                      ribo = rownames(sce)[ribo_genes])

lapply(feature_ctrls, head)
$mito
[1] "MT-ND1_ENSG00000198888"  "MT-ND2_ENSG00000198763" 
[3] "MT-CO1_ENSG00000198804"  "MT-CO2_ENSG00000198712" 
[5] "MT-ATP8_ENSG00000228253" "MT-ATP6_ENSG00000198899"

$ribo
[1] "RPL22_ENSG00000116251"   "RPL11_ENSG00000142676"  
[3] "RPS6KA1_ENSG00000117676" "RPS8_ENSG00000142937"   
[5] "RPL5_ENSG00000122406"    "RPS27_ENSG00000177954"  
MT-ND1_ENSG00000198888

MT-ND2_ENSG00000198763

MT-CO1_ENSG00000198804

MT-CO2_ENSG00000198712

MT-ATP8_ENSG00000228253

MT-ATP6_ENSG00000198899
RPL22_ENSG00000116251

RPL11_ENSG00000142676

RPS6KA1_ENSG00000117676

RPS8_ENSG00000142937

RPL5_ENSG00000122406

RPS27_ENSG00000177954

And call the calcualteQCMetrics function in scater:

sce <- calculateQCMetrics(sce, feature_controls = feature_ctrls)
datatable(head(as.data.frame(colData(sce))))

What to look for

My personal favourite plot to QC scRNA-seq data:

plotColData(sce, x = "total_features_by_counts", y = "pct_counts_mito")

Typically retain cells that have < 10% mitochondrial transcripts and > 1000 features, but this is dataset dependent - for example, tumour cells typically have higher metabolic burden, leading to higher % mitochondrial (we typically use 20% as a filter then).

plotPCA(sce, colour_by = "pct_counts_mito")
Warning: 'add_ticks' is deprecated.
Use '+ geom_rug(...)' instead.

For this going to use a simple threshold of 10% mitochondrial. Importantly, we re-compute the QC metrics, size factors and normalization

mito_thresh <- 10

sce_qc <- sce[, sce$pct_counts_mito < mito_thresh]

sce_qc <- scran::computeSumFactors(sce_qc, BPPARAM = MulticoreParam(10))

sce_qc <- normalize(sce_qc)

sce_qc <- calculateQCMetrics(sce_qc, feature_controls = feature_ctrls)
sce_qc <- runPCA(sce_qc)
sce_qc <- runUMAP(sce_qc)
# plotPCA(sce_qc, colour_by = "total_features_by_counts")
plotUMAP(sce_qc, colour_by = "pct_counts_mito")
Warning: 'add_ticks' is deprecated.
Use '+ geom_rug(...)' instead.

plotScater(sce)

plotScater(sce_qc)

plotHighestExprs(sce_qc)

Some useful plotting functions in Scater

Reduced dimension plots

I can call

  • plotPCA(sce, colour_by = "x")
  • plotUMAP(sce, colour_by = "x")
  • plotTSNE(sce, colour_by = "x")

where x is:

  • Any column of colData(sce) (= the cell specific data) to colour by metadata
  • Any gene name in rownames(sce) to colour by expression
plotPCA(sce, colour_by = "SAA1_ENSG00000173432")
Warning: 'add_ticks' is deprecated.
Use '+ geom_rug(...)' instead.

Additional plots

plotColData(sce_qc, x = "total_counts", y = "pct_counts_mito")

plotExpression(sce_qc, 
               x = "total_counts", 
               features = "GAPDH_ENSG00000111640")

Using CellAssign to assign cells to known types

CellAssign is our new method to assign cells to known cell types. It relies on assuming cell types over-express their own markers, e.g. an epithelial tumour cell should overexpress EPCAM relative to other cell types.

library(cellassign)

In this example, the data we have just performed QC and exploratory analysis of is liver cells, that we expect to contain a certain number of cell types. To begin, we specify a list, where each item corresponds to a set of marker genes for a given cell type:

liver_marker_list <- list(
        Hepatocytes = c("ALB", "HAMP", "ARG1", "PCK1", "AFP", "BCHE"), 
        LSECs = c("CALCRL", "FCGR2B", "VWF"),
        Cholangiocytes = c("KRT19", "EPCAM", "CLDN4", "CLDN10", "SOX9", "MMP7", "CXCL1", "CFTR", "TFF2", "KRT7", "CD24"), 
        `Hepatic Stellate Cells` = c("ACTA2", "COL1A1", "COL1A2", "COL3A1", "DCN", "MYL9"),
        Macrophages = c("CD68", "MARCO", "FCGR3A", "LYZ", "PTPRC"),
        `ab T cells` = c("CD2", "CD3D", "TRAC", "IL32", "CD3E", "PTPRC"),
        `gd T cells` = c("NKG7", "FCGR3A", "HOPX", "GNLY", "KLRF1", "CMC1", "CCL3", "PTPRC"),
        `NK cells` = c("GZMK", "KLRF1", "CCL3", "CMC1", "NKG7", "PTPRC"),
        `Plasma cells` = c("CD27", "IGHG1", "CD79A", "IGHG2", "PTPRC", "IGKC"),
        `Mature B cells` = c("MS4A1", "LTB", "CD52", "IGHD", "CD79A", "PTPRC", "IGKC"),
        `Erythroid cells` = c("HBB", "SLC25A37", "CA1", "ALAS2")
)

To begin, we use cellassign’s marker_list_to_mat function to convert this into a (binary) cell type by marker matrix:

mgi <- marker_list_to_mat(liver_marker_list, include_other = FALSE)

pheatmap(mgi)

We then make sure all of these markers exist in our SingleCellExperiment:

marker_in_sce <- match(rownames(mgi), rowData(sce_qc)$Symbol)
stopifnot(all(!is.na(marker_in_sce)))

And finally we subset sce to just the marker genes:

sce_marker <- sce_qc[marker_in_sce, ]
stopifnot(all.equal(rownames(mgi), rowData(sce_marker)$Symbol))

We then call cellassign passing in the SingleCellExperiment, marker info, the size factors we’ve calculated, as well as various other options:

save.image("~/Desktop/image.rds")
counts(sce_marker) <- as.matrix(counts(sce_marker))

print(dim(sce_marker))
[1]  56 331
print(dim(mgi))
[1] 56 11
fit <- cellassign(
  exprs_obj = sce_marker,
  marker_gene_info = mgi,
  s = sizeFactors(sce_qc),
  shrinkage = TRUE,
  max_iter_adam = 50,
  min_delta = 2,
  verbose = TRUE
)
50  L old: -45341.4248612881; L new: -13086.9923911386; Difference (%): 0.71136786214427
50  L old: -13086.9923911386; L new: -12637.7934535153; Difference (%): 0.0343240772362165
50  L old: -12637.7934535153; L new: -12459.5540075771; Difference (%): 0.0141036840484733
50  L old: -12459.5540075771; L new: -12394.0136869888; Difference (%): 0.00526024611703001
50  L old: -12394.0136869888; L new: -12369.1578771128; Difference (%): 0.00200546897104786
50  L old: -12369.1578771128; L new: -12359.4793407949; Difference (%): 0.000782473343297033
50  L old: -12359.4793407949; L new: -12352.0271620321; Difference (%): 0.000602952483459608
50  L old: -12352.0271620321; L new: -12347.4391063071; Difference (%): 0.000371441518452011
50  L old: -12347.4391063071; L new: -12344.007432408; Difference (%): 0.000277925962584415
20  L old: -12344.007432408; L new: -12342.7594546598; Difference (%): 0.000101099886328378
20  L old: -12342.7594546598; L new: -12341.5678489241; Difference (%): 9.65428954589848e-05
fit
A cellassign fit for 331 cells, 56 genes, 11 cell types with 0 covariates
           To access MLE cell types, call x$cell_type
           To access MLE parameter estimates, call x$mle_params

Add the cell types to the sce:

sce_qc$cell_type <- fit$cell_type
plotUMAP(sce_qc, colour_by = "cell_type")
Warning: 'add_ticks' is deprecated.
Use '+ geom_rug(...)' instead.

acol <- data.frame(`cellassign cell type` = sce_qc$cell_type)
rownames(acol) <- colnames(sce_qc)

pheatmap(as.matrix(logcounts(sce_marker)),
         annotation_col = acol)

pheatmap(fit$mle_params$gamma)

LS0tCnRpdGxlOiAiVXNpbmcgUiAmIEJpb2NvbmR1Y3RvciB0byBhc3NpZ24gY2VsbCB0eXBlcyB0byBzaW5nbGUtY2VsbCBSTkEtc2VxIGRhdGEiCm91dHB1dDogCiAgaHRtbF9ub3RlYm9vazoKICAgIHRvYzogdHJ1ZQogICAgdG9jX2Zsb2F0OiB0cnVlCmF1dGhvcjogS2llcmFuIFIgQ2FtcGJlbGwgKGtpY2FtcGJlbGxAYmNjcmMuY2EpCmRhdGU6ICJgciBmb3JtYXQoU3lzLnRpbWUoKSwgJyVkICVCLCAlWScpYCIKCi0tLQoKIyBPdmVydmlldyBvZiB0aGlzIHR1dG9yaWFsCgpBZnRlciBnb2luZyB0aHJvdWdoIHRoaXMgdHV0b3JpYWwgeW91IHNob3VsZCBiZSBhYmxlIHRvOgoKMS4gUmVhZCAxMFggc2luZ2xlLWNlbGwgUk5BLXNlcSBkYXRhIGludG8gYFNpbmdsZUNlbGxFeHBlcmltZW50YCBvYmplY3RzCjIuIEtub3cgaG93IHRvIG5hdmlnYXRlIGBTaW5nbGVDZWxsRXhwZXJpbWVudGAgb2JqZWN0cwozLiBCYXNpYyBRQyBvZiBzaW5nbGUtY2VsbCBSTkEtc2VxIGRhdGEgdXNpbmcgYHNjYXRlcmAKNC4gQ3JlYXRlIGxvdyBkaW1lbnNpb25hbCBwbG90cyAoUENBLCBVTUFQLCB0U05FKQo1LiBBc3NpZ24gY2VsbHMgdG8ga25vd24gY2VsbCB0eXBlcyB1c2luZyBgY2VsbGFzc2lnbmAKCgojIyBSZXNvdXJjZXMKCiogW1NpbmdsZUNlbGxFeHBlcmltZW50IHZpZ25ldHRlXShodHRwczovL2Jpb2NvbmR1Y3Rvci5vcmcvcGFja2FnZXMvcmVsZWFzZS9iaW9jL3ZpZ25ldHRlcy9TaW5nbGVDZWxsRXhwZXJpbWVudC9pbnN0L2RvYy9pbnRyby5odG1sKQoqIFtTY2F0ZXIgdmlnbmV0dGVdKGh0dHBzOi8vYmlvY29uZHVjdG9yLm9yZy9wYWNrYWdlcy9kZXZlbC9iaW9jL3ZpZ25ldHRlcy9zY2F0ZXIvaW5zdC9kb2MvdmlnbmV0dGUtaW50cm8uaHRtbCkKKiBbQ2VsbEFzc2lnbiBSIHBhY2thZ2VdKGh0dHBzOi8vZ2l0aHViLmNvbS9pcnJhdGlvbm9uZS9jZWxsYXNzaWduKQoqIFtDZWxsQXNzaWduIHByZXByaW50XShodHRwOi8vYml0Lmx5L2NlbGxhc3NpZ25wcmVwcmludCkKKiBbSGVtYmVyZyBsYWIgc2NSTkEtc2VxIGNvdXJzZV0oaHR0cHM6Ly9oZW1iZXJnLWxhYi5naXRodWIuaW8vc2NSTkEuc2VxLmNvdXJzZS9pbmRleC5odG1sKQoqIFtPcmNoZXN0cmF0aW5nIFNpbmdsZS1DZWxsIEFuYWx5c2lzIHdpdGggQmlvY29uZHVjdG9yXShodHRwOi8vb3NjYS5iaW9jb25kdWN0b3Iub3JnLykKCioqVGhpcyBkb2N1bWVudCBjb21waWxlZDoqKiBodHRwOi8va2llcmFucmNhbXBiZWxsLmdpdGh1Yi5pby9yLXdvcmtzaG9wLW1hcmNoLTIwMTkKCiMjIFJlcXVpcmVkIHBhY2thZ2VzCgpgYGB7ciBjYWNoZSA9IEZBTFNFfQpzdXBwcmVzc1BhY2thZ2VTdGFydHVwTWVzc2FnZXMoewogIGxpYnJhcnkoc2NhdGVyKSAjIEJpb0NvbmR1Y3RvcgogIGxpYnJhcnkoU2luZ2xlQ2VsbEV4cGVyaW1lbnQpICMgQmlvQ29uZHVjdG9yCiAgbGlicmFyeShEcm9wbGV0VXRpbHMpICMgQmlvQ29uZHVjdG9yCiAgbGlicmFyeSh0aWR5dmVyc2UpICMgQ1JBTgogIGxpYnJhcnkoaGVyZSkgIyBDUkFOCiAgbGlicmFyeShEVCkgIyBDUkFOCiAgbGlicmFyeShwaGVhdG1hcCkgIyBDUkFOCn0pCgprbml0cjo6b3B0c19jaHVuayRzZXQoZWNobyA9IFRSVUUsCiAgICAgICAgICAgICAgICAgICAgICBjYWNoZSA9IFRSVUUpCmBgYAoKSWYgeW91IG5lZWQgdG8gaW5zdGFsbCBhbnkgb2YgdGhlc2UsIHlvdSBjYW4gZG8gc28gYnk6CgpgYGB7ciBldmFsPUZ9Cmluc3RhbGwucGFja2FnZXMoYygidGlkeXZlcnNlIiwgImhlcmUiLCAiRFQiLCAicGhlYXRtYXAiLCAiQmlvY01hbmFnZXIiKSkKCkJpb2NNYW5hZ2VyOjppbnN0YWxsKGMoInNjYXRlciIsICJTaW5nbGVDZWxsRXhwZXJpbWVudCIsICJEcm9wbGV0VXRpbHMiKSkKYGBgCgoqKkl0IGlzIGhpZ2hseSByZWNvbW1lbmRlZCB5b3UgZG8gc28gYmVmb3JlIHRoZSB0dXRvcmlhbCBhcyB0aGlzIGNhbiB0YWtlIHNpZ25pZmljYW50IHRpbWUhKioKCiMgSW5zdGFsbGF0aW9uIG9mIGNlbGxhc3NpZ24KClRvIGluc3RhbGwgY2VsbGFzc2lnbiwgd2UgbmVlZCB0byBpbnN0YWxsIEdvb2dsZSdzIFRlbnNvcmZsb3cgZnJhbWV3b3JrLiAqKklmIHRoaXMgZG9lc24ndCB3b3JrIGluIHRoZSB0dXRvcmlhbCAtIGRvbid0IHdvcnJ5LCB5b3Ugd29uJ3QgbmVlZCBpdCBmb3IgODAlIG9mIHdoYXQgd2UgY292ZXIqKi4KCmBgYHtyLCBldmFsPVR9Cmluc3RhbGwucGFja2FnZXMoInRlbnNvcmZsb3ciLCByZXBvcyA9ICJodHRwOi8vY3Jhbi51cy5yLXByb2plY3Qub3JnIikKdGVuc29yZmxvdzo6aW5zdGFsbF90ZW5zb3JmbG93KCkKCmluc3RhbGwucGFja2FnZXMoImRldnRvb2xzIiwgcmVwb3MgPSAiaHR0cDovL2NyYW4udXMuci1wcm9qZWN0Lm9yZyIpICMgSWYgbm90IGFscmVhZHkgaW5zdGFsbGVkCmRldnRvb2xzOjppbnN0YWxsX2dpdGh1YigiSXJyYXRpb25vbmUvY2VsbGFzc2lnbiIpCgpgYGAKCgojIFJlYWQgaW4gMTBYIHNjUk5BLXNlcSBkYXRhCgpXZSdyZSBnb2luZyB0byB1c2Ugc29tZSByZS1wcm9jZXNzZWQgZGF0YSBmcm9tIFtTaW5nbGUgY2VsbCBSTkEgc2VxdWVuY2luZyBvZiBodW1hbiBsaXZlciByZXZlYWxzIGRpc3RpbmN0IGludHJhaGVwYXRpYyBtYWNyb3BoYWdlIHBvcHVsYXRpb25zXShodHRwczovL3d3dy5uYXR1cmUuY29tL2FydGljbGVzL3M0MTQ2Ny0wMTgtMDYzMTgtNykgKHBhdGllbnQgMSBzcGVjaWZpY2FsbHkpLiBJZiB5b3UgZ2l0IGNsb25lZCB0aGlzIHJlcG9zaXRvcnkgKGh0dHA6Ly9naXRodWIuY29tL2tpZXJhbnJjYW1wYmVsbC9odHRwczovL2dpdGh1Yi5jb20va2llcmFucmNhbXBiZWxsL3Itd29ya3Nob3AtbWFyY2gtMjAxOSkgdGhpcyBjYW4gYmUgZm91bmQgaW4gdGhlIGRpcmVjdG9yeSBgZGF0YS9vdXRzL2ZpbHRlcmVkX2dlbmVfYmNfbWF0cmljZXMvR1JDaDM4YDoKCmBgYHtyfQpkYXRhX2RpciA8LSBoZXJlKCJkYXRhIiwgIm91dHMiLCAiZmlsdGVyZWRfZ2VuZV9iY19tYXRyaWNlcyIsICJHUkNoMzgiKQoKcHJpbnQoZGF0YV9kaXIpCgpwcmludChkaXIoZGF0YV9kaXIpKQpgYGAKCjEwWCBkYXRhIHR5cGljYWxseSBoYXMgYSBiYXJjb2RlcyBmaWxlIChpbmRpY2F0aW5nIGNlbGwgYmFyY29kZXMpLCBhIGdlbmVzIGZpbGUgKGluZGljYXRpbmcgYSBtYXBwaW5nIGJldHdlZW4gZ2VuZXMgYW5kIGluZGljZXMpIGFuZCB0aGUgYWN0dWFsIGV4cHJlc3Npb24gZGF0YSBhcyAqKnJhdyBjb3VudHMqKiBpbiBgbWF0cml4Lm10eGAuCgpXZSBjYW4gcmVhZCB0aGVzZSBpbiB0byBhIGBTaW5nbGVDZWxsRXhwZXJpbWVudGAgb2JqZWN0IHVzaW5nIHRoZSBgcmVhZDEweENvdW50c2AgZnVuY3Rpb24KCmBgYHtyfQpzY2UgPC0gcmVhZDEweENvdW50cyhkYXRhX2RpcikKCnNjZQpgYGAKCiMgQSBicmllZiB0b3VyIG9mIHRoZSBgU2luZ2xlQ2VsbEV4cGVyaW1lbnRgIHBhY2thZ2UKCk92ZXJhbGwgaWRlYTogdGhpbmsgb2YgeW91ciBjb3VudCBtYXRyaXggd2hlcmUgdGhlICoqY29sdW1ucyBhcmUgY2VsbHMqKiBhbmQgdGhlICoqcm93cyBhcmUgZ2VuZXMqKi4KClNvIHdoZW4geW91IHNlZSB0aGluZ3MgbGlrZSBgcm93RGF0YWAgdGhpbmsgYGdlbmVEYXRhYCwgYW5kIGBjb2xEYXRhYCB0aGluayBgY2VsbERhdGFgIQoKCgoKCiMjIERhdGEgZGltZW5zaW9ucwoKR2V0dGluZyBkYXRhIGRpbWVuc2lvbnMKCmBgYHtyfQpucm93KHNjZSkgIyBOdW1iZXIgb2Ygcm93cyA9IGdlbmVzCm5jb2woc2NlKSAjIE51bWJlciBvZiBjb2x1bW5zID0gY2VsbHMKYGBgCgpgYGB7cn0KcHJpbnQoc2NlKQpgYGAKCgojIyBTdWJzZXR0aW5nCgpgYGB7cn0Kc2NlWywgYygxLDMsNSldICMgU3Vic2V0IHRvIGNlbGxzIDEsIDMsIDUKYGBgCgpgYGB7cn0Kc2NlW2MoMiw0LDYpLCBdICMgU3Vic2V0IHRvIGdlbmVzIDIsIDQsIDYgCmBgYAoKIyMgRmVhdHVyZSBhbmQgY2VsbCBtZXRhZGF0YQoKYHJvd25hbWVzYCA9IGdlbmUgbmFtZXMKCmBgYHtyfQpoZWFkKHJvd25hbWVzKHNjZSkpCmBgYAoKYGNvbG5hbWVzYCA9IGNlbGwgbmFtZXMgPSBiYXJjb2RlcyAoc29tZXRpbWVzKQoKYGBge3J9CmhlYWQoY29sbmFtZXMoc2NlKSkKYGBgCgoKCgpDb2x1bW4gZGF0YSAoY2VsbCBzcGVjaWZpYyBtZXRhZGF0YSkKCmBgYHtyfQpoZWFkKGNvbERhdGEoc2NlKSkKYGBgCgpJIG1pZ2h0IHdhbnQgdG8gc2V0IHRoZSBjb2x1bW4gbmFtZXMgdG8gdGhlIGJhcmNvZGU6CgpgYGB7cn0KY29sbmFtZXMoc2NlKSA8LSBjb2xEYXRhKHNjZSkkQmFyY29kZQoKaGVhZChjb2xuYW1lcyhzY2UpKQpgYGAKCkFuZCBJIGNhbiBzdWJzZXQgb24gYmFyY29kZToKCmBgYHtyfQpzY2VbLCAiQUFBQ0NUR0NBR1RBQUdDRy0xIl0KYGBgCgoKClJvdyBkYXRhIChnZW5lIHNwZWNpZmljIG1ldGFkYXRhKQoKYGBge3J9CmhlYWQocm93RGF0YShzY2UpKQpgYGAKCgpgcmVkdWNlZERpbXNgIGlzIHdoZXJlIG91ciBQQ0EsVU1BUCx0U05FIHJlcHJlc2VudGF0aW9ucyB3aWxsIGxpdmUgLSBidXQgd2UgaGF2ZW4ndCBtYWRlIHRoZW0geWV0CgpgYGB7cn0KcmVkdWNlZERpbXMoc2NlKQpgYGAKCgpgc2l6ZUZhY3RvcnNgIGlzIHdoZXJlIG91ciBjZWxsIHNpemUgZmFjdG9ycyB3aWxsIGxpdmUgLSBidXQgd2UgaGF2ZW4ndCBjYWxjdWxhdGVkIHRoZW0geWV0CgpgYGB7cn0KaGVhZChzaXplRmFjdG9ycyhzY2UpKQpgYGAKCgpUaGUgYWJpbGl0eSB0byBoYXZlIG11bHRpcGxlIGBhc3NheXNgIGlzIG9uZSBvZiB0aGUgdW5pcXVlIGFkdmFudGFnZXMgb2YgdGhlIFNpbmdsZUNlbGxFeHBlcmltZW50IGFwcHJvYWNoIC0gSSBjYW4gY2FycnkgYXJvdW5kIGBjb3VudHNgLCBgbG9nY291bnRzYCwgYW5kIGFueSBvdGhlciB3ZWlyZCBkYXRhIHRyYW5zZm9ybWF0aW9uIEkgbWlnaHQgbGlrZS4gUmlnaHQgbm93IHdlIG9ubHkgaGF2ZSByYXcgY291bnRzLCBiZWNhdXNlIHRoYXQncyB3aGF0IHdlJ3ZlIHJlYWQgaW46CgpgYGB7cn0KbmFtZXMoYXNzYXlzKHNjZSkpCmBgYAoKYGBge3IsIGV2YWwgPSBGQUxTRX0KYXNzYXkoc2NlLCAiY291bnRzIikKY291bnRzKHNjZSkKYGBgCgoKSSBjYW4gbWFrZSBteSBvd246CgpgYGB7cn0KYXNzYXkoc2NlLCAibXlfc3VwZXJfc3RyYW5nZV9hc3NheSIpIDwtIHNpbihhcy5tYXRyaXgoY291bnRzKHNjZSkpKQoKbmFtZXMoYXNzYXlzKHNjZSkpCmBgYAoKYGBge3J9CmNsYXNzKGFzc2F5KHNjZSwgIm15X3N1cGVyX3N0cmFuZ2VfYXNzYXkiKSkKYGBgCgpOb3RlIHRoZSBiZWF1dHkgb2YgU2luZ2xlQ2VsbEV4cGVyaW1lbnRzIGlzIHRoYXQgc3Vic2V0dGluZyBpcyBjb25zaXN0ZW50OiBpZiBJIHdhbnQgdG8gc3Vic2V0IG9ubHkgc29tZSBjZWxscyBhbmQgZ2VuZXM6CgpgYGB7cn0Kc2NlX3N1YnNldCA8LSBzY2VbYygxLDMsNSksIGMoMiw0LDYsOCldCmBgYAoKVGhlbiBldmVyeXRoaW5nIGVsc2UgaXMgc3Vic2V0IHRvbyEKCmBgYHtyfQpwcmludChkaW0oY291bnRzKHNjZV9zdWJzZXQpKSkKcHJpbnQobGVuZ3RoKHNpemVGYWN0b3JzKHNjZV9zdWJzZXQpKSkKcHJpbnQoZGltKHJvd0RhdGEoc2NlX3N1YnNldCkpKQpgYGAKClNvIHRoZSBhcHByb2FjaCBtYXkgc2VlbSBsaWtlIGEgbG90IG9mIHdvcmsgdXAgZnJvbnQgdG8ganVzdCBjYXJyeSBhcm91bmQgYSBjb3VudCBtYXRyaXggYW5kIHNvbWUgbWV0YWRhdGEsIGJ1dCB0aGlzIHNvcnQgb2YgY29uc2lzdGVudCBzdWJzZXR0aW5nIG1ha2VzIGl0IF9tdWNoXyBoYXJkZXIgKGJ1dCBzdGlsbCBub3QgaW1wb3NzaWJsZSkgdG8gaW50cm9kdWNlIGJ1Z3MgaW50byB5b3VyIGFuYWx5c2lzLgoKCgojIFF1YWxpdHkgY29udHJvbCBvZiBzY1JOQS1zZXEgZGF0YQoKIyMgR2V0dGluZyBzdGFydGVkCgpGaXJzdCB3ZSBkbyBzb21lIGtleSB0aGluZ3MgdG8gb3VyIGRhdGE6CgoxLiBHZXQgc29tZSBleHRyYSBnZW5lIGRhdGEsIGluY2x1ZGluZyB0aGUgY2hyb21vc29tZSBuYW1lCjIuIENvbXB1dGUgdGhlIHNpemUgZmFjdG9ycwozLiBDb21wdXRlIG5vcm1hbGl6ZWQgbG9nIGNvdW50cyAKCmBgYHtyfQpyb3dEYXRhKHNjZSkkZW5zZW1ibF9nZW5lX2lkIDwtIHJvd25hbWVzKHNjZSkKICAKc2NlIDwtIGdldEJNRmVhdHVyZUFubm9zKHNjZSwgCiAgICAgICAgICAgICAgICAgICAgICAgICBmaWx0ZXJzID0gImVuc2VtYmxfZ2VuZV9pZCIsCiAgICAgICAgICAgICAgICAgICAgICAgICBhdHRyaWJ1dGVzID0gYygiZW5zZW1ibF9nZW5lX2lkIiwgImhnbmNfc3ltYm9sIiwgImVudHJlemdlbmUiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgInN0YXJ0X3Bvc2l0aW9uIiwgImVuZF9wb3NpdGlvbiIsICJjaHJvbW9zb21lX25hbWUiKSwKICAgICAgICAgICAgICAgICAgICAgICAgIGRhdGFzZXQgPSAiaHNhcGllbnNfZ2VuZV9lbnNlbWJsIikKCiAgCiMgQ2FsY3VsYXRlIHNpemUgZmFjdG9ycwpzY2UgPC0gc2NyYW46OmNvbXB1dGVTdW1GYWN0b3JzKHNjZSwgQlBQQVJBTSA9IE11bHRpY29yZVBhcmFtKDEwKSkKCgojIENvbXB1dGUgbG9nIG5vcm1hbCBleHByZXNzaW9uIHZhbHVlcwpzY2UgPC0gbm9ybWFsaXplKHNjZSkKCmBgYAoKYGBge3J9Cm5hbWVzKGFzc2F5cyhzY2UpKQpgYGAKCmBgYHtyfQpoZWFkKHNpemVGYWN0b3JzKHNjZSkpCmBgYAoKCmBgYHtyfQpzY2UgPC0gcnVuUENBKHNjZSkKc2NlIDwtIHJ1blVNQVAoc2NlKQpgYGAKCmBgYHtyfQpyZWR1Y2VkRGltcyhzY2UpCmBgYAoKSnVzdCBnaXZlIG1lIG15IFBDQSEKCmBgYHtyfQpoZWFkKHJlZHVjZWREaW1zKHNjZSlbWydQQ0EnXV0pCmBgYAoKCkkgbGlrZSB0byBhZGQgdGhlIHN5bWJvbHMgdG8gdGhlIHJvd25hbWVzOgoKYGBge3J9CnJvd25hbWVzKHNjZSkgPC0gcGFzdGUwKHJvd0RhdGEoc2NlKSRTeW1ib2wsICJfIiwgcm93bmFtZXMoc2NlKSkKaGVhZChyb3duYW1lcyhzY2UpKQpgYGAKCgpXZSBuZXh0IG5lZWQgdG8gd29yayBvdXQgd2hpY2ggZ2VuZXMgYXJlIG1pdG9jaG9uZHJpYWwgYW5kIHJpYm9zb21hbCBhcyB0aGVzZSB3b3JrIHdlbGwgZm9yIFFDOgoKYGBge3J9CiMgR2V0IE1pdG9jaG9uZHJpYWwgZ2VuZXMgZm9yIFFDOgptdF9nZW5lcyA8LSB3aGljaChyb3dEYXRhKHNjZSkkY2hyb21vc29tZV9uYW1lID09ICJNVCIpCnJpYm9fZ2VuZXMgPC0gZ3JlcGwoIl5SUFtMU10iLCByb3dEYXRhKHNjZSkkU3ltYm9sKQpmZWF0dXJlX2N0cmxzIDwtIGxpc3QobWl0byA9IHJvd25hbWVzKHNjZSlbbXRfZ2VuZXNdLAogICAgICAgICAgICAgICAgICAgICAgcmlibyA9IHJvd25hbWVzKHNjZSlbcmlib19nZW5lc10pCgpsYXBwbHkoZmVhdHVyZV9jdHJscywgaGVhZCkKYGBgCgpBbmQgY2FsbCB0aGUgYGNhbGN1YWx0ZVFDTWV0cmljc2AgZnVuY3Rpb24gaW4gc2NhdGVyOgoKYGBge3J9CnNjZSA8LSBjYWxjdWxhdGVRQ01ldHJpY3Moc2NlLCBmZWF0dXJlX2NvbnRyb2xzID0gZmVhdHVyZV9jdHJscykKYGBgCgpgYGB7cn0KZGF0YXRhYmxlKGhlYWQoYXMuZGF0YS5mcmFtZShjb2xEYXRhKHNjZSkpKSkKYGBgCgoKIyMgV2hhdCB0byBsb29rIGZvcgoKTXkgcGVyc29uYWwgZmF2b3VyaXRlIHBsb3QgdG8gUUMgc2NSTkEtc2VxIGRhdGE6CgpgYGB7cn0KcGxvdENvbERhdGEoc2NlLCB4ID0gInRvdGFsX2ZlYXR1cmVzX2J5X2NvdW50cyIsIHkgPSAicGN0X2NvdW50c19taXRvIikKYGBgCgpUeXBpY2FsbHkgcmV0YWluIGNlbGxzIHRoYXQgaGF2ZSA8IDEwJSBtaXRvY2hvbmRyaWFsIHRyYW5zY3JpcHRzIGFuZCA+IDEwMDAgZmVhdHVyZXMsIGJ1dCB0aGlzIGlzICoqZGF0YXNldCBkZXBlbmRlbnQqKiAtIGZvciBleGFtcGxlLCB0dW1vdXIgY2VsbHMgdHlwaWNhbGx5IGhhdmUgaGlnaGVyIG1ldGFib2xpYyBidXJkZW4sIGxlYWRpbmcgdG8gaGlnaGVyICUgbWl0b2Nob25kcmlhbCAod2UgdHlwaWNhbGx5IHVzZSAyMCUgYXMgYSBmaWx0ZXIgdGhlbikuCgpgYGB7cn0KcGxvdFBDQShzY2UsIGNvbG91cl9ieSA9ICJwY3RfY291bnRzX21pdG8iKQpgYGAKCkZvciB0aGlzIGdvaW5nIHRvIHVzZSBhIHNpbXBsZSB0aHJlc2hvbGQgb2YgMTAlIG1pdG9jaG9uZHJpYWwuIEltcG9ydGFudGx5LCB3ZSByZS1jb21wdXRlIHRoZSBRQyBtZXRyaWNzLCBzaXplIGZhY3RvcnMgYW5kIG5vcm1hbGl6YXRpb24KCmBgYHtyfQptaXRvX3RocmVzaCA8LSAxMAoKc2NlX3FjIDwtIHNjZVssIHNjZSRwY3RfY291bnRzX21pdG8gPCBtaXRvX3RocmVzaF0KCnNjZV9xYyA8LSBzY3Jhbjo6Y29tcHV0ZVN1bUZhY3RvcnMoc2NlX3FjLCBCUFBBUkFNID0gTXVsdGljb3JlUGFyYW0oMTApKQoKc2NlX3FjIDwtIG5vcm1hbGl6ZShzY2VfcWMpCgpzY2VfcWMgPC0gY2FsY3VsYXRlUUNNZXRyaWNzKHNjZV9xYywgZmVhdHVyZV9jb250cm9scyA9IGZlYXR1cmVfY3RybHMpCmBgYAoKYGBge3J9CnNjZV9xYyA8LSBydW5QQ0Eoc2NlX3FjKQpzY2VfcWMgPC0gcnVuVU1BUChzY2VfcWMpCmBgYAoKYGBge3J9CiMgcGxvdFBDQShzY2VfcWMsIGNvbG91cl9ieSA9ICJ0b3RhbF9mZWF0dXJlc19ieV9jb3VudHMiKQpwbG90VU1BUChzY2VfcWMsIGNvbG91cl9ieSA9ICJwY3RfY291bnRzX21pdG8iKQpgYGAKCmBgYHtyfQpwbG90U2NhdGVyKHNjZSkKcGxvdFNjYXRlcihzY2VfcWMpCmBgYAoKCmBgYHtyfQpwbG90SGlnaGVzdEV4cHJzKHNjZV9xYykKYGBgCgojIyBTb21lIHVzZWZ1bCBwbG90dGluZyBmdW5jdGlvbnMgaW4gU2NhdGVyCgojIyMgUmVkdWNlZCBkaW1lbnNpb24gcGxvdHMKCkkgY2FuIGNhbGwgCgoqIGBwbG90UENBKHNjZSwgY29sb3VyX2J5ID0gIngiKWAKKiBgcGxvdFVNQVAoc2NlLCBjb2xvdXJfYnkgPSAieCIpYAoqIGBwbG90VFNORShzY2UsIGNvbG91cl9ieSA9ICJ4IilgCgp3aGVyZSBgeGAgaXM6CgoqIEFueSBjb2x1bW4gb2YgYGNvbERhdGEoc2NlKWAgKD0gdGhlIGNlbGwgc3BlY2lmaWMgZGF0YSkgdG8gY29sb3VyIGJ5IG1ldGFkYXRhCiogQW55IGdlbmUgbmFtZSBpbiBgcm93bmFtZXMoc2NlKWAgdG8gY29sb3VyIGJ5IGV4cHJlc3Npb24KCmBgYHtyfQpwbG90UENBKHNjZSwgY29sb3VyX2J5ID0gIlNBQTFfRU5TRzAwMDAwMTczNDMyIikKYGBgCgojIyMgQWRkaXRpb25hbCBwbG90cwoKYGBge3J9CnBsb3RDb2xEYXRhKHNjZV9xYywgeCA9ICJ0b3RhbF9jb3VudHMiLCB5ID0gInBjdF9jb3VudHNfbWl0byIpCmBgYAoKYGBge3J9CnBsb3RFeHByZXNzaW9uKHNjZV9xYywgCiAgICAgICAgICAgICAgIHggPSAidG90YWxfY291bnRzIiwgCiAgICAgICAgICAgICAgIGZlYXR1cmVzID0gIkdBUERIX0VOU0cwMDAwMDExMTY0MCIpCmBgYAoKCgojIFVzaW5nIENlbGxBc3NpZ24gdG8gYXNzaWduIGNlbGxzIHRvIGtub3duIHR5cGVzCgpDZWxsQXNzaWduIGlzIG91ciBuZXcgbWV0aG9kIHRvIGFzc2lnbiBjZWxscyB0byBrbm93biBjZWxsIHR5cGVzLiBJdCByZWxpZXMgb24gYXNzdW1pbmcgY2VsbCB0eXBlcyBvdmVyLWV4cHJlc3MgdGhlaXIgb3duIG1hcmtlcnMsIGUuZy4gYW4gZXBpdGhlbGlhbCB0dW1vdXIgY2VsbCBzaG91bGQgb3ZlcmV4cHJlc3MgRVBDQU0gcmVsYXRpdmUgdG8gb3RoZXIgY2VsbCB0eXBlcy4gCgpgYGB7cn0KbGlicmFyeShjZWxsYXNzaWduKQpgYGAKCgoKSW4gdGhpcyBleGFtcGxlLCB0aGUgZGF0YSB3ZSBoYXZlIGp1c3QgcGVyZm9ybWVkIFFDIGFuZCBleHBsb3JhdG9yeSBhbmFseXNpcyBvZiBpcyBsaXZlciBjZWxscywgdGhhdCB3ZSBleHBlY3QgdG8gY29udGFpbiBhIGNlcnRhaW4gbnVtYmVyIG9mIGNlbGwgdHlwZXMuIFRvIGJlZ2luLCB3ZSBzcGVjaWZ5IGEgbGlzdCwgd2hlcmUgZWFjaCBpdGVtIGNvcnJlc3BvbmRzIHRvIGEgc2V0IG9mIG1hcmtlciBnZW5lcyBmb3IgYSBnaXZlbiBjZWxsIHR5cGU6CgpgYGB7cn0KbGl2ZXJfbWFya2VyX2xpc3QgPC0gbGlzdCgKICAgICAgICBIZXBhdG9jeXRlcyA9IGMoIkFMQiIsICJIQU1QIiwgIkFSRzEiLCAiUENLMSIsICJBRlAiLCAiQkNIRSIpLCAKICAgICAgICBMU0VDcyA9IGMoIkNBTENSTCIsICJGQ0dSMkIiLCAiVldGIiksCiAgICAgICAgQ2hvbGFuZ2lvY3l0ZXMgPSBjKCJLUlQxOSIsICJFUENBTSIsICJDTERONCIsICJDTEROMTAiLCAiU09YOSIsICJNTVA3IiwgIkNYQ0wxIiwgIkNGVFIiLCAiVEZGMiIsICJLUlQ3IiwgIkNEMjQiKSwgCiAgICAgICAgYEhlcGF0aWMgU3RlbGxhdGUgQ2VsbHNgID0gYygiQUNUQTIiLCAiQ09MMUExIiwgIkNPTDFBMiIsICJDT0wzQTEiLCAiRENOIiwgIk1ZTDkiKSwKICAgICAgICBNYWNyb3BoYWdlcyA9IGMoIkNENjgiLCAiTUFSQ08iLCAiRkNHUjNBIiwgIkxZWiIsICJQVFBSQyIpLAogICAgICAgIGBhYiBUIGNlbGxzYCA9IGMoIkNEMiIsICJDRDNEIiwgIlRSQUMiLCAiSUwzMiIsICJDRDNFIiwgIlBUUFJDIiksCiAgICAgICAgYGdkIFQgY2VsbHNgID0gYygiTktHNyIsICJGQ0dSM0EiLCAiSE9QWCIsICJHTkxZIiwgIktMUkYxIiwgIkNNQzEiLCAiQ0NMMyIsICJQVFBSQyIpLAogICAgICAgIGBOSyBjZWxsc2AgPSBjKCJHWk1LIiwgIktMUkYxIiwgIkNDTDMiLCAiQ01DMSIsICJOS0c3IiwgIlBUUFJDIiksCiAgICAgICAgYFBsYXNtYSBjZWxsc2AgPSBjKCJDRDI3IiwgIklHSEcxIiwgIkNENzlBIiwgIklHSEcyIiwgIlBUUFJDIiwgIklHS0MiKSwKICAgICAgICBgTWF0dXJlIEIgY2VsbHNgID0gYygiTVM0QTEiLCAiTFRCIiwgIkNENTIiLCAiSUdIRCIsICJDRDc5QSIsICJQVFBSQyIsICJJR0tDIiksCiAgICAgICAgYEVyeXRocm9pZCBjZWxsc2AgPSBjKCJIQkIiLCAiU0xDMjVBMzciLCAiQ0ExIiwgIkFMQVMyIikKKQpgYGAKClRvIGJlZ2luLCB3ZSB1c2UgYGNlbGxhc3NpZ25gJ3MgYG1hcmtlcl9saXN0X3RvX21hdGAgZnVuY3Rpb24gdG8gY29udmVydCB0aGlzIGludG8gYSAoYmluYXJ5KSBjZWxsIHR5cGUgYnkgbWFya2VyIG1hdHJpeDoKCmBgYHtyLCBmaWcud2lkdGggPSA1LCBmaWcuaGVpZ2h0ID0gMTB9Cm1naSA8LSBtYXJrZXJfbGlzdF90b19tYXQobGl2ZXJfbWFya2VyX2xpc3QsIGluY2x1ZGVfb3RoZXIgPSBGQUxTRSkKCnBoZWF0bWFwKG1naSkKYGBgCgpXZSB0aGVuIG1ha2Ugc3VyZSBhbGwgb2YgdGhlc2UgbWFya2VycyBleGlzdCBpbiBvdXIgU2luZ2xlQ2VsbEV4cGVyaW1lbnQ6CgpgYGB7cn0KbWFya2VyX2luX3NjZSA8LSBtYXRjaChyb3duYW1lcyhtZ2kpLCByb3dEYXRhKHNjZV9xYykkU3ltYm9sKQpzdG9waWZub3QoYWxsKCFpcy5uYShtYXJrZXJfaW5fc2NlKSkpCmBgYAoKQW5kIGZpbmFsbHkgd2Ugc3Vic2V0IGBzY2VgIHRvIGp1c3QgdGhlIG1hcmtlciBnZW5lczoKCmBgYHtyfQpzY2VfbWFya2VyIDwtIHNjZV9xY1ttYXJrZXJfaW5fc2NlLCBdCmBgYAoKYGBge3J9CnN0b3BpZm5vdChhbGwuZXF1YWwocm93bmFtZXMobWdpKSwgcm93RGF0YShzY2VfbWFya2VyKSRTeW1ib2wpKQpgYGAKCldlIHRoZW4gY2FsbCBgY2VsbGFzc2lnbmAgcGFzc2luZyBpbiB0aGUgYFNpbmdsZUNlbGxFeHBlcmltZW50YCwgbWFya2VyIGluZm8sIHRoZSBzaXplIGZhY3RvcnMgd2UndmUgY2FsY3VsYXRlZCwgYXMgd2VsbCBhcyB2YXJpb3VzIG90aGVyIG9wdGlvbnM6CgpgYGB7cn0Kc2F2ZS5pbWFnZSgifi9EZXNrdG9wL2ltYWdlLnJkcyIpCmBgYAoKCmBgYHtyIHdhcm5pbmcgPSBGLCBtZXNzYWdlID0gRn0KY291bnRzKHNjZV9tYXJrZXIpIDwtIGFzLm1hdHJpeChjb3VudHMoc2NlX21hcmtlcikpCgpwcmludChkaW0oc2NlX21hcmtlcikpCnByaW50KGRpbShtZ2kpKQoKZml0IDwtIGNlbGxhc3NpZ24oCiAgZXhwcnNfb2JqID0gc2NlX21hcmtlciwKICBtYXJrZXJfZ2VuZV9pbmZvID0gbWdpLAogIHMgPSBzaXplRmFjdG9ycyhzY2VfcWMpLAogIHNocmlua2FnZSA9IFRSVUUsCiAgbWF4X2l0ZXJfYWRhbSA9IDUwLAogIG1pbl9kZWx0YSA9IDIsCiAgdmVyYm9zZSA9IFRSVUUKKQoKYGBgCgoKCmBgYHtyfQpmaXQKCmBgYAoKCkFkZCB0aGUgY2VsbCB0eXBlcyB0byB0aGUgc2NlOgoKYGBge3J9CnNjZV9xYyRjZWxsX3R5cGUgPC0gZml0JGNlbGxfdHlwZQoKYGBgCgpgYGB7cn0KcGxvdFVNQVAoc2NlX3FjLCBjb2xvdXJfYnkgPSAiY2VsbF90eXBlIikKCmBgYAoKCmBgYHtyLCBmaWcud2lkdGggPSA5LCBmaWcuaGVpZ2h0ID0gMTB9CmFjb2wgPC0gZGF0YS5mcmFtZShgY2VsbGFzc2lnbiBjZWxsIHR5cGVgID0gc2NlX3FjJGNlbGxfdHlwZSkKcm93bmFtZXMoYWNvbCkgPC0gY29sbmFtZXMoc2NlX3FjKQoKcGhlYXRtYXAoYXMubWF0cml4KGxvZ2NvdW50cyhzY2VfbWFya2VyKSksCiAgICAgICAgIGFubm90YXRpb25fY29sID0gYWNvbCkKCmBgYAoKYGBge3J9CnBoZWF0bWFwKGZpdCRtbGVfcGFyYW1zJGdhbW1hKQoKYGBgCgoKCg==